Regression testing is a technique for discovering bugs that may have been introduced into software code as an unintended result of changes made to the code. This sort of testing is typically initiated after a programmer has attempted to fix a recognized bug or otherwise changed or added code to an existing program. For this purpose, programmers often develop a suite of tests, in the form of inputs, commands and execution parameters, for example, that are applied to exercise various parts of the code after a change has been made.
Software code coverage analysis is commonly used to track areas of a program that are exercised by a set of tests (and thus to find areas that are not exercised). A code coverage analyzer is a software tool that determines which lines of code were executed in a given test run. It can be used to give a quantitative measure of code coverage, as well as to assist in creating additional test cases to increase coverage and identify redundant test cases that do not increase coverage. For example, Bullseye Testing Technology (Redmond, Wash.) offers the BullseyeCoverage code coverage analyzer for C++ and C, which can be used to evaluate various coverage measures, including line coverage, statement coverage, branch coverage, condition coverage and function coverage. Rosenblum et al. describe a method for choosing an efficient regression test suite based on coverage analysis in “Using Coverage Information to Predict the Cost-Effectiveness of Regression Testing Strategies,” IEEE Transactions on Software Engineering 23:3 (March, 1997), pages 146-156.
Another use of coverage analysis in test suite generation is described in U.S. Patent Application Publication 2005/0044533. A current software build is compared to a reference software build, which is typically a known, previous build. The comparison identifies those areas in the current software build that have changed with regard to the reference software build. The identified areas are used by a coverage analysis process to determine a focused test suite to test the modified areas of the current build.
U.S. Patent Application Publication 2005/0223361 describes another method for non-redundant software testing, based on identifying changes in the execution paths of a software program. Test cases capable of traversing the changed or new execution paths are identified and executed to test the modified code.
In contrast to regression testing, software change impact analysis provides a forecast of the potential effects of changes before the changes are implemented. Impact analysis, in other words, estimates what will be affected in software and related documentation if a proposed software change is made. For example, U.S. Patent Application Publications 2006/0117310 describes a method for analyzing the impact of an update to a software system. The impact analysis identifies resources that are affected by the update, such as configuration information and executable code in dynamic link libraries.